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SYSTEMS AND METHODS FOR ILLUMINANT ESTIMATION 
TECHNICAL FIELD 

[01] The present invention relates generally to digital image processing and more particularly 
to methods and systems for estimation of an image illuminant. 

BACKGROUND 

[02] Colors viewed in an image are dependent on the light that illuminates the subject of the 
image. Different illuminants will cause different light to be reflected from surfaces of the image 
subject matter. The human visual system approximately corrects these variations in reflected 
light such that surfaces remain approximately color constant. However, when images are 
captured on media and viewed under a light source different than the source in the imaged scene, 
these natural corrections do not take place. Accordingly, it is often desirable for recorded images 
to be color-balanced to a standard, reference light source in order to appear as they would to the 
natural eye. This balancing or color correction can be performed once the scene illuminant is 
identified. 

[03] Estimating the color parameters of an image illuminant is an important first step in color 
correcting a digital image. In a strict model-matching scheme, the accuracy of the estimation is 
often limited by the size of the model base. This is because illuminant parameter values are 
explicitly available only at the predefined model positions and because the match procedure itself 
typically reduces to selection of the "best-fit" model from within a limited set. Increasing the 
number of models can improve accuracy. However, additional memory is required to store the 
larger set of models and additional time is required to match against all models in the larger 
model base. 

[04] Under known methods, illuminant estimation employs a fixed set of known illuminants, 
which are each characterized by the gamut of color values that are possible under that illuminant. 
A color histogram is computed for the image and is compared with each of the model histograms 
using some form of match metric such as intersection, correlation, minimum distance, etc. The 
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match metric is used to select the model that best accounts for the image data from the set of 
illuminants. 



SUMMARY 

[05] Embodiments of the present invention increase illuminant or colorbalance correction 
estimation accuracy by fitting an analytic form to the match surface derived from initial 
comparisons of the image data against the model base. The analytic form may then be used to 
interpolate between model data. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[06] The present embodiments will become more fully apparent from the following 

description and appended claims, taken in conjunction with the accompanying drawings. 

Understanding that these drawings depict only typical embodiments and are, therefore, not to be 

considered limiting of the invention's scope, the embodiments will be described with additional 

specificity and detail through use of the accompanying drawings in which: 

Fig. la is a diagram illustrating a set of candidate illuminants as x-y chromaticity 
coordinates; and 

Fig. 2 is a diagram of an exemplary match score surface. 
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DETAILED DESCRIPTION 

[07] Illuminant estimation may be approached through a model matching strategy. In such a 
regime, a fixed set of illuminants is modeled. Modeling may be performed parametrically, by 
sample statistics or by other methods. The model that best accounts for the image data is chosen 
as the scene illuminant most likely to have produced the image. This decision process relies on 
computing a similar parametric or statistical description from the image and then performing a 
matching procedure of the image description with respect to the model base in order to select the 
"best" model. 

[08] An exemplary model set, shown in Figure 1, consists of 81 illuminants plotted in x-y 
chromaticity space. Chromaticity coordinates are shown as dots 2. In this particular example, 
these coordinates have been regularly sampled in CIE-Lab coordinates centered on the D65 white 
point 4, which is a daylight reference illuminant. The coordinates are then mapped to x-y 
chromaticity space as displayed in Figure 1. 

[09] In some embodiments of the present invention, a match surface is calculated to estimate a 
likelihood of each illuminant being the illuminant of a particular image. In some embodiments, 
each image element, such as a pixel, is compared to the series of model illuminants to determine 
a match score, which is an indicator of the likelihood of being the illuminant of that element of 
the image. 

[10] An exemplary match surface is shown in Figure 2, which is a plot of x and y chromaticity 
and the match score. In this plot, the chromaticity of each candidate or model illuminant is 
plotted on the horizontal axes while the vertical axis represents the likelihood of being the image 
illuminant or match score. In some embodiments, a fixed set of illuminants, which may or may 
not occupy a conventional grid, identifies the horizontal position of a point of the match surface. 
An analysis of each illuminant in the fixed set with respect to image data then identifies the z- 
axis coordinate of the surface point. Once the surface points are identified, an analytic form may 
be matched to the surface. In some embodiments, we may assume an over-determined system. 
In some embodiments and for some image types, a quadratic form works well, however, other 
orders of surfaces, such as cubic and quartic may be used. 
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[11] In some embodiments of the present invention a biquadratic surface may be used, which 
can be specified by the following implicit form: 

/ (x, y )= Ax 2 + Bxy + Cy 2 + Dx + Ey + F . 
[12] The value of this form at the x-y coordinates of each model illuminant is given by: 

AV = Z, 

[13] where A is the so-called design matrix and V is a vector of surface parameters. In some 
embodiments, V is chosen so as to minimize the error between this analytic form and the 
corresponding match surface values. 

[14] One well-known technique for minimizing the sum of squared errors for such a system is 

the pseudoinverse method. Note that because the chromaticity coordinates of the model set are all 

determined a priori, the pseudoinverse matrix can be computed entirely offline. Some 

embodiments of the present invention use this method, which comprises the following steps. 

[15] Generally steps 1 and 2 will be performed offline, however, they may be performed 

online as well when resources and time constraints allow. 

1) Form the design matrix for the predetermined set of model illuminants, based on each 
model's chromaticity coordinates. This can be a matrix of monomial basis functions 
in the chromaticity coordinates of each illuminant. For a quadratic form it is defined 
as follows: 



r 2 
x \ 








y\ 


A 


x 2 y 2 


A 


x 2 




A 


x n y n 


yl 


x n 





2) Compute the Moore-Penrose pseudoinverse of the design matrix. For a quadratic form 
this is a 6 x N matrix where N is the number of illuminant models. It is defined as: 

e = (a t a) _1 a t 

[16] The following steps are generally performed online. 

3) Form the image histogram and compute the corresponding match surface over the 
model set (e.g. as shown in Figure 2). The vector of match scores corresponding to 
the N illuminants may be designated as follows: 
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Z = 



Xl 



4) Find the minimum value xl of Z and the chromaticity coordinates x min = [x m y m f of 
the associated illuminant. 

5) Compute the parameter vector V = [a b C D e Ff for the best-fit least-squares 
approximating surface as follows: 

V = EZ 

The best fit surface is then given as: 

V T X, with X= [k 2 xy y 2 x y if 

Other surface fit methods may be used in other embodiments. 



6) Form the partial derivatives of the resulting surface with respect to x and y and set 
these equal to zero, since these derivatives vanish at the extremum. 

df 

-^- = 2Ax + By + D = 0 
dx 



dy 



= Bx + 2Cy + E = 0 



In matrix notation this can be written simply as 

Jx = K or 
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7) Solve for the chromaticity coordinates x = [x yf of the minimum point of the analytic 
form. 
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In alternative embodiments, maximum points may be used to determine the best 



match. 



8) If the resulting point is closer to the reference illuminant than minimum model point 
x min > se * the scene illuminant to coordinates of the analytic minimum, x 0 = x ; 
otherwise set it to the coordinates of the minimum-distance model, x 0 = x min . This 
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selection criterion always chooses the more conservative correction, which may be a 
good strategy in practice. However, 

[17] In some embodiments, once the chromaticity coordinates of the scene iUuminant are 
estimated, the corresponding correction matrix can be computed in any of a number of ways. For 
example, if a linear model of surface reflectances and illuminants is used, one might first 
transform the scene iUuminant xyY coordinates (* 0 y 0 i) T to the corresponding tri-stimulus 
XYZ value x 0 = (x 0 Y 0 Z 0 f and then use the following equations to derive the corresponding 
correction matrix C for the scene illuminant. 

^£0=(TB £ )- 1 x o 
E 0 =diag(B £ u> £0 ) 
E ref =diag^pd re/ ) 
C = TE re/ B $ (TE 0 B s y l 

where 

T represents a matrix of color matching functions or sensor sensitivities 

B E represents an illuminant spectral power linear model 

diag represents an operator that converts a vector to a diagonal matrix 

spd re f represents a reference illuminant spectral power distribution 

B s represents a surface reflectance linear model 
[18] In other embodiments, other selection criteria can be used. In some embodiments, the 
analytic minimum may be selected if it occurs within the hull of the model set. If the analytic 
minimum does not fall within the hull of the model set, the minimum model value may be 
selected. In still other embodiments, exterior analytic minima can be projected along the line to 
the daylight reference and the intersection of this line with the hull of the model set can be the 
selected point. 

[19] In other embodiments color coordinates may be represented in a three dimensional 
colorspace, with corresponding three-dimensional image and model color distributions and 
corresponding higher dimensional design matrices. 

[20] . In some embodiments, the analytic form can be resolved to an arbitrary accuracy by first 
employing differential geometry to locate the extremum of the form and then using interpolation 
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to derive the corresponding illuminant parameters or colorbalance correction parameters for this 
location. 

[21] Embodiments of the present invention also offer the possibility of decreasing the size of 
the model base and consequently decreasing the time and/or resources required for the match 
step. This is because the accuracy of the illuminant parameter estimation is not directly 
constrained by the number of models. Only enough models to ensure the stability of the analytic 
fit are required. 

[22] Algorithms of embodiments of the present invention may be implemented in software on 
a general-purpose computer or on a special-purpose computing device such as a DSP. 
Embodiments may also be implements by dedicated circuitry such as an ASIC. Embodiment 
processes may be implemented in any image processing pipeline that outputs an image for 
display, for retrieval, for indexing or for other purposes. 



[23] What is claimed is: 



